Skip to content

Fix Update Dependencies failing when two jobs push to main at once - #584

Draft
cratis-stagehand[bot] wants to merge 1 commit into
mainfrom
fix/update-dependencies-push-race
Draft

Fix Update Dependencies failing when two jobs push to main at once#584
cratis-stagehand[bot] wants to merge 1 commit into
mainfrom
fix/update-dependencies-push-race

Conversation

@cratis-stagehand

Copy link
Copy Markdown

Important

This branch is incomplete and must not be merged as-is. The accompanying commit to .github/workflows/update-dependencies.yml could not be pushed — the bot account (cratis-stagehand[bot]) has no workflows permission, so both git push and the contents API reject it:

refusing to allow a GitHub App to create or update workflow
`.github/workflows/update-dependencies.yml` without `workflows` permission

The exact patch is in the comment on #582. Someone with workflows permission needs to apply it to this branch, after which the PR is complete and can be marked ready.

Summary

"Update Dependencies" has failed on and off for weeks — 5 of the last 15 runs, most recently run 33171239370. The dependency updates were never the problem. The three update-npm matrix jobs run in parallel and each commits and pushes straight to main, so whichever pushes second loses the race:

! [remote rejected] main -> main (cannot lock ref 'refs/heads/main':
  is at a466a8ea... but expected 024970fb...)
error: failed to push some refs to 'https://github.com/Cratis/Samples'

fail-fast then cancelled the sibling jobs, so a single lost race took the whole run down and silently dropped the bumps the cancelled jobs had already computed. Every one of the recent failures is this same rejected push.

Fixed

Changed

Verification

The race and the fix were reproduced end to end locally against real git repositories, using depth-1 clones to mirror actions/checkout@v4:

  1. Reproduced the failure — two jobs clone, both commit, one pushes, the second's git pull --rebase && git push is rejected with failed to push some refs. Same failure mode as the CI run.
  2. Proved the fix — under the identical race, push-with-retry.sh is rejected on attempt 1, rebases onto the new head, and succeeds on attempt 2. Both folders' updates end up on the remote; neither is lost.
  3. Proved it still fails loudly — with an unreachable remote it exits non-zero and emits a ::error:: annotation, so a genuine push problem is not swallowed by the retry.

Repository gates on this branch:

  • dotnet build Samples.slnx -c Debug — 0 errors (18 warnings, all pre-existing on main; this PR touches no C#)
  • dotnet build Samples.slnx -c Release -p:CratisProxiesOutputPath= — 0 errors
  • CI's focused spec set — all green (Backend 4, CrossStore 5, MultiTenancy 9, OperationsDiagnosis 2, Processing 3)

This also disproves the original hypothesis on #582 that a dependency update had broken the build or tests: main builds and tests clean at the current package versions.

Not verified

  • The workflow file change has not run in CI, because it could not be pushed (see the blocker above). It is validated only by YAML parse and by the local reproduction of the push behavior.
  • The Library Testcontainers specs were not run — they need Docker, which is unavailable in this environment. CI excludes them from its spec set for the same reason.

The dependency update jobs commit straight to main from several jobs at
once, so a push rejected with "cannot lock ref 'refs/heads/main'" means
another job simply moved the branch first. Rebase onto the new remote
head and try again rather than treating it as a failure.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants